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ABSTRACT 

When  we  look  around  a  natural  environment  the  eye  is 
directed  toward  objects  we  select.  We  seem  to  do  this  task 
accurately  and  effortlessly.  Yet,  even  such  a  simple  task 
presents  real  problems  for  the  oculomotor  system,  namely,  to 
select  the  relevant  target  from  the  detailed  background  so  that 
only  information  contained  in  it  influences  the  line  of  sight  and 
to  spatlally-pool  information  in  the  selected  target  so  that  the 
line  of  sight  lands  at  a  single  position  within  the  selected 
target.  We  have  found  that:  (1)  the  saccadic  target  is 
designated  by  means  of  selective  perceptual  attention,  which 
means  people  cannot  prepare  to  look  to  one  target  while 
accurately  perceiving  targets  elsewhere,  and  (2)  there  is  a 
highly-accurate  spatial  pooling  process  wh^-ch  can  direct  the  line 
of  sight  to  precise  positions  within  large  targets.  The  results 
show  that  the  oculomotor  system  is  capable  of  extremely  rapid  and 
effective  scanning.  The  procedures  humans  use  to  accomplish  this 
task  may  prove  useful  for  the  guidance  of  robotic  systems  which 
need  to  move  about  in  patterned  visual  environments. 
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Summary  of  goals  and  approach 


Human  beings  are  confronted  by  far  more  visual  information 
than  they  can  deal  with  in  a  single  glance.  One  of  the  chief 
limitations  on  our  visual  abilities  is  our  acuity:  We  can  only 
discern  fine  details  of  images  located  in  a  small,  central 
retinal  region  known  as  the  fovea.  For  this  reason,  scanning  eye 
movements  are  critical  for  apprehending  even  ordinary  visual 
scenes.  Their  importance  increases  dramatically  for  visually- 
demanding  tasks,  such  as  reading  or  search,  where  enormous 
amounts  of  fine  detail  are  presented  at  one  time. 

There  are  two  components  to  scanning  eye  movements.  First, 
the  observer  has  to  decide  where  to  look.  Second,  she  has  to  be 
able  to  execute  the  eye  movement  that  carries  out  the  decision. 
Our  research  focuses  on  the  second  question.  We  want  to  know  how 
the  decisions  are  translated  into  action. 

We  can  specify  this  problem  more  precisely.  Suppose  you 
decide  to  look  at  the  face  of  someone  seated  nearby.  Your  chosen 
target,  the  face,  is  large,  but  the  line  of  sight  must  land  at 
one  particular  place  within  it.  Yet,  we  are  not  usually  not 
aware  of  choosing  the  precise  landing  position  (the  tip  of  the 
nose,  perhaps).  How  is  the  landing  position  determined? 

We  have  been  studying  a  laboratory  version  of  this  task  in 
which  observers  have  to  look  at  a  large,  but  simple,  eccentric 
target,  such  as  an  outline  drawing  of  a  circle  or  a  cluster  of 
random  dots.  We  found  that,  providing  the  intent  is  to  "look  at 
the  form  as  a  whole",  rather  than  at  a  particular  place  within 
it,  the  line  of  sight  lands,  reliably,  near  the  center-of -gravity 
of  the  form.  Moreover,  we  can  increase  the  size  of  the  target 
from  a  small  point  to  a  large  form  and  neither  the  accuracy  nor 
the  precision  of  the  saccadic  landing  positions  suffer.  This 
tells  us  that  there  is  some  internal  process  that  computes  the 
central  landing  position  by  pooling  information  over  space  in  an 
effortless,  automatic  fashion.  We  are  now  studying  more 
elaborate  target  configurations  (spatially-filtered  images,  for 
example)  in  collaboration  with  researchers  at  SRI  in  Princeton 
(Bergen  and  Lubin)  to  find  out  what  processing  steps  the  visual 
system  uses  to  pool  spatial  information  and  compute  the  central 
landing  position.  The  pooling  may,  for  example,  be  carried  out 
only  on  some  of  the  information  in  the  target,  such  as  its  edges 
or  perhaps  its  lower  spatial  frequency  components. 

There  is  a  second  component  to  our  research.  Accurate  and 
precise  aiming  of  the  line  of  sight  to  the  chosen  targets 
requires  that  the  spatial  pooling  referred  to  above  encompasses 
only  visual  information  in  the  selected  visual  targets. 

Irrelevant  background  information  should  not  be  included.  One 
way  that  we  might  do  this  is  by  making  use  of  our  capacity  to 
shift  spatial  attention  to  some  portions  of  the  visual  field  at 
the  expense  of  others.  "Spatial  attention"  refers  to  our 
internal  processing  resources,  which  can  be  assigned,  at  will,  to 
different  objects  or  different  locations.  It  is  well  known  that 
allocation  of  attention  affects  perceptual  abilities.  We  have 
shown  that  it  affects  saccades  as  well.  We  found  that  perceptual 
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recogntion  is  enhanced,  albeit  briefly,  at  the  goal  of  the 
saccade  right  before  the  saccade  is  to  occur.  We  did  this  in 
experiments  that  required  subjects  to  try  to  identify  a  target  in 
one  place  while  at  the  same  time  programming  the  saccade 
elsewhere.  We  found  they  couldn't  do  both  tasks  at  the  same  time 
as  well  as  they  could  do  them  one  at  a  time:  either  saccadic 
latency  or  perceptual  identification  suffered.  Surprisingly, 
modest  increases  in  saccadic  latency  were  sufficient  to  allow 
accurate  perceptual  identification  of  a  target  remote  from  the 
saccadic  goal.  We  draw  two  conclusions  from  these  results. 

First,  that  saccades  require  some  shift  of  spatial  attention  to 
the  target,  and  this  is  how  we  direct  the  line  of  sight  toward 
chosen  details  without  it  being  dragged  off  in  the  direction  of 
irrelevant  details.  Second,  the  attentional  requirements  of 
saccades  are  relatively  modest.  In  a  brief  fixation  pause 
between  saccades  we  may  be  able  to  mentally  inspect  several 
locations  in  the  visual  display,  perhaps  evaluating  which  we  will 
look  at,  with  no  important  cost  in  either  saccadic  or  visual 
performance. 

There  are  three  long-range  implications  of  these  results: 

First,  by  specifying  both  the  attentional  requirements  of 
saccades  and  the  processing  steps  of  spatial  pooling,  we 
facilitate  the  attempts  of  neurophysiologists  to  understand  how 
the  brain  performs  these  tasks.  In  particular,  our  results  may 
aid  in  the  interpretation  of  single  unit  recordings  from  neural 
sites  known  to  be  active  before  saccades. 

Second,  we  find  both  in  the  spatial  pooling  and  the 
saccades/attention  task  surprisingly  good  performance  with 
relatively  little  effort  exerted  by  the  subject.  Our  oculomotor 
system  appears  to  be  well  suited  to  handle  the  demands  of  even 
complex  scanning  tasks.  This  means  the  primary  limit  on 
effective  processing  will  be  cognitive  (target  selection  and 
interpretation)  and  not  visual-motor.  Our  visual  motor  system  is 
an  excellent  tool  to  accomplish  the  required  tasks. 

Third,  the  same  problems  that  confront  humans  who  have  to 
scan  visual  displays  also  confront  robots.  To  the  extent  that 
attentional  allocation  and  spatial  pooling  provide  useful 
solutions  for  humans,  these  same  processing  steps  may  lead  to  new 
effective  solutions  for  guidance  of  robotic  scanning  systems. 
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Specific  accomplishnients  and  work  in  progress; 


Saccadic  accuracy  and  precision;  We  studied  saccades  to 
single  point  and  form  targets  in  an  attempt  to  establish  the  best 
possible  accuracy  and  precision  of  saccades.  Study  of  such  a 
basic  question  was  necessary  because  the  of  the  prevailing  belief 
that  saccades  are  inaccurate,  undershooting  the  target  by  about 
10%.  We  found  no  such  undershoots  provided  that  subjects  are 
instructed  to  be  accurate.  Moreover,  accuracy  and  precision  were 
remarkably  good,  with  saccades  on  average  missing  the  target  by 
only  1%  of  eccentricity  and  SD's  of  landing  position  only  6%  (a 
precision  that  is  similar  to  one  of  our  most  precise  perceptual 
skills,  namely,  judgment  of  relative  target  location). 
Surprisingly,  neither  accuracy,  precision  nor  latency  were 
impaired  by  large  increases  in  target  size,  showing  that  there 
must  be  a  spatial  pooling  mechanism  that  computes,  with  great 
precision,  a  central  landing  position  within  eccentric  objects. 
(Kowler  and  Blaser,  1994) 


Saccades  and  attention;  We  looked  at  the  link  between 
shifts  of  attention  and  shifts  of  the  line  of  sight.  Our  goal 
was  to  find  out  whether  attention  must  be  drawn  to  the  locus  of 
the  saccade,  or,  alternatively,  whether  it  is  possible  to  attend 
one  object  and  simultaneously  prepare  to  look  somewhere  else.  We 
performed  several  "dual-task"  experiments  (concurrent  measures  of 
saccades  and  perception)  in  which  subjects  had  to  look  at  a 
target  (a  letter  in  a  ring  of  8  letters)  while  at  the  same  time 
identifying  another  letter  at  the  designated  locus  of  the  saccade 
or  at  another  location.  The  display  disappeared  before  the 
saccade  occurred  to  guarantee  no  change  in  the  retinal 
eccentricity  of  the  letters.  Letter  identification  was  better  at 
the  saccadic  goal  than  elsewhere.  But  surprisingly, 
identification  at  the  non-goal  locations  could  be  improved 
substantially  with  little  or  no  cost  to  either  saccadic  latency 
or  accuracy.  Reaching  optimal  levels  of  perceptual 
identification  required  latency  increases  of  50-75  msec.  Thus, 
saccades  require  allocation  of  attention  to  the  target,  but  the 
attentional  requirements  of  a  saccade  are  modest.  The  results 
suggest  a  dissociation  between  systems  that  determine  the 
saccadic  goal  (controlled  by  means  of  selective  attention)  and 
those  that  issue  the  "go"  signal  to  trigger  the  saccade.  One  way 
for  these  two  systems  to  work  together  as  effectively  as  our 
results  show  they  can  would  be  to  allow  the  triggering  system 
access  to  information  about  the  current  locus  of  attention  and, 
in  addition,  to  another  signal  specificing  the  intended  saccadic 
goal.  This  would  allow  the  saccade  to  be  launched  as  soon  as  the 
locus  of  attention  coincides  with  the  pre-selected  goal.  Such  a 
scheme  would  allow  rapid  and  accurate  scanning  while  minimizing 
the  need  for  time-consuming  decisions  "on-line"  about  when  and 
where  to  aim  the  eye.  (Kowler,  Anderson  and  Dosher,  1994). 

Saccades  to  dot  clusters;  We  studied  the  accuracy  and 
precision  of  saccades  to  small  (60')  clusters  of  random  dots. 

Our  goal  was  to  begin  to  characterize  the  spatial  pooling 
process,  which  we  believe  (He  and  Kowler,  1989,  1991)  computes 
the  saccadic  endpoint  automatically  within  the  region  of  spatial 
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attention.  In  these  experiments  both  the  spatial  position  of  the 
outermost  boundary  of  the  cluster  and  the  center  of  gravity  of 
the  cluster  were.' vAried  independently.  We  found  that  saccades 
were  directed*  with  great  precision  to  the  center  of  gravity  of 
the  cluster.  These  results  suggest  an  averaging  process  that 
pools  information  in  a  selected  target  and  does  not  give  special 
weight  to  outer  boundaries.  We  are  continuing  this  study  using 
random  dot  fields  in  order  to  map  the  size  and  shape  of  the 
region  oyer  .which.  aut;gma,titj  pooling  can  occur,  in  effect, 

finding  an  'bchipmbtgr  ,  "r^peptitVf’^jh^eld"  analogous  to  visual 
receptive  f i’e'lds  (Kowl^r,  Shariuf ,,  ;McGowan  and  Chubb). 

!  Piiltbfed^-’stiiftull':  Another  way  to  characterize  the  spatial 
pooling  proce94»'^ii  ''tb  study^  stirjui'i  with  specially  chosen  spatial 
characteristics.  We  plan  to '\i.sa . stimuli  that  are  filtered  so 
that  they  contain spatial  frequency.  We  want  to 
know  whether  the  spatial-pooling  process  is  restricted  to  only 
certain  frequency  bands.  If  so,  then  removal  of  these 
frequencies  should  disturb  saccadic  accuracy  and  precision.  We 
will  also  vary  stimulus  contrast.  Insensitivity  to  contrast  over 
a  broad  range  suggest  some  mechanism  must  exist  that  compares  the 
output  of  a  number  of  similarly-tuned  filters.  These  experiments 
are  in  an  early  stage  and  are  being  carried  out  in  collaboration 
with  Drs.  James  Bergen  and  Jeff  Lubin  at  the  Sarnoff  Research 
Laboratory,  SRI,  Princeton,  NJ.  (Sharma,  Lubin,  Bergen  and 
Kowler) . 

The  attentional  field;  These  experiments  follow  the  work  on 
saccades  and  attention  described  above  and  they  represent  an 
attempt  to  better  characterize  the  spatial  limits  of  attentional 
regions.  We  designate  two  eccentric  locations  via  central  cues 
and  then  test  perceptual  identification  for  targets  at  these 
locations.  If  there  are  limits  to  the  size  of  an  attentional 
region,  then  the  perceptual  performance  will  be  better  when  the 
two  cues  fall  in  the  same  attentional  region.  Alternatively, 
attention  may  be  "object-based"  in  which  case  the  spatial  locus 
of  the  cues  will  not  be  important.  These  experiments  also 
provide  a  perceptual  analog  to  the  experiments  on  saccades  and 
attention  described  above  in  that  they  will  allow  us  to  find  out 
whether  the  costs  of  dividing  attention  between  a  perceptual  and 
saccadic  target  also  obtain  when  attention  is  divided  between 
two  perceptual  targets.  (Bahcall,  Anderson,  and  Kowler) 
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